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(57) ABSTRACT 

A method and apparatus for communicating information in 
a network is described. A packet for the information is 
generated at a first network device. The first network device 
assigns a flow label to the packet. The flow label indicates 
that the packet is part of a particular sequence of packets. 
The first network device also assigns a direction to the 
packet by, for example, setting a bit in the flow label. The 
packet is then sent lo a second network device through at 
least one intermediate network device. This process is 
continued for the entire sequence of packets. The interme- 
diate network device actuaUy routes the packets to the 
second network device. The intermediate network device 
receives the packets at an input port. A flow label is 
identified for each packet. The intermediate network device 
determines whether a flow table has an entry for the flow 
label. If there is no present entry for the flow label in the flow 
table, an entry for the flow label is created. If there is an 
entry for the flow label, an output port associated with the 
flow label is obtained. The intermediate network device then 
sends the packet to the output port. This continues at each 
intermediate network device until each packet reaches the 
second network device. 

15 Claims^ 2 Drawing Sheets 
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METHOD AND APPARATUS FOR 
PROVIDING QUALITY OF SERVICE USING 
THE INTERNET PROTOCOL 

CROSS-REFERENCE TO RELATED 
APPUCAnON 

This application claims the benefit of U.S. Provisional 
AppHcaUon No. 60/081,479, filed Apr. 10, 1998, cntided 
"Provision of Quality Services Using the Internet Protocol," 
the entire disclosure of which is hereby incorporated by 
reference. 

HELD OF THE INVENTION 

The embodiments of the invention relate to commimica- 
tions in general. In particular, the embodiments of the 
invention relate to a method and apparatus for providing 
quality of service using the Internet Protocol (IP). 

BACKGROUND OF THE INVENTION 

With the explosive growth of the Internet and new net- 
work applications almost exclusively written for the Internet 
Protocol (IP), it has become essential to optimize protocols 
and network management for the IP. Originally designed for 
data networking, the Internet is increasingly being used for 
audio and video applications. Whereas using a single net- 
work level technology may potentially simplify network 
management, providing sufficient service quality for multi- 
media applications over the Internet remains a significant 
challenge. After almost a decade of emphasis on resource 
reservations and end-to-end Quality of Service (QoS), as 
part of both the design of Asynchronous Transfer Mode 
(ATM) networks and standardization of the Resource Res- 
ervation Protocol (RSVP), there is now a significant back- 
lash against these state rich, fine grained QoS models. This 
is in part based on the observation that rather than unavail- 
ability of bandwidth, service instability is causing problems 
to multimedia applications. Current efforts on differentiated 
services are an attempt to develop a service model that 
improves the service quality of the Internet while acting at 
aggregate levels. However ensuring stable service level 
requires richer traffic management facilities than currently 
available in the Internet. 

The two most essential characteristics of the IP that have 
contributed to its success and distinguish it from connection 
oriented networks arc the softness of state inside the 
network, and the aggregation properties of this state. Apart 
from the routing database, for best effort destination based 
routing, (cached) state is used purely for performance 
enhancement, but is not essential for correctly delivering 
packets to destination. In particular this state can be lost, or 
removed at routers discretion without affecting the validity 
of state elsewhere in the network. With single class desti- 
nation based routing, prefix matching effectively aggregates 
the forwarding information for multiple destinations into a 
single entry per prefix. Even with nodal service differentia- 
tion (ToS bits) this property is retained. 

In contrast, the strengths of the state rich telephony 
network, and derived connection-oriented models (e.g., 
ATM), are their service quality assurances. In part, the 
quality assurances are achieved through resource reserva- 
tions and tight channel scheduling, based on declared or 
inferred user objectives. In part, the consistent quality is 
achieved through network management; exploiting intra- 
domain knowledge about network load and conditions. In 
addition connection oriented models enhance stability, as the 
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time-scale of (load) change becomes that of connection 
diu'ation, rather than that of routing updates and packet 
interarrivals. Whereas traditional methods exploit mecha- 
nisms for connectivity and reservations to achieve quality of 

S service and implement traffic management, the coupling of 
these distinct mechanisms has contributed to the perceived 
complexity of connection oriented networks. 

In view of the foregoing, it can be appreciated that a 
substantial need exists for introducing the QoS advantages 

10 of connection-oriented networks into connectionless net- 
works (e.g., using IP) without losing the advantages given 
by connectionless networks. 

SUMMARY OF THE INVENTION 

One embodiment of the invention comprises a method 
and apparatus for communicating information in a network. 
A packet for the information is generated at a first network 
device such as an end system. The first network device 

^ assigns a flow label to the packet. The flow label indicates 
that the packet is part of a particular sequence of packets. 
The first network device also assigns a direction to the 
packet by, for example, setting a bit in the flow label. The 
packet is then sent to a second network device (e.g., another 

^ end system) through at least one intermediate network 
device (e.g., a router or switch). This process is continued 
for the entire sequence of packets for a given flow. 

The intermediate network device actually routes the pack- 
ets to the second network device. The intermediate network 

30 device receives the packets at an input port. A flow label is 
identified for each packet. The intermediate network device 
determines whether a flow table has an entry for the flow 
label. If there is no present entry for the flow label in the flow 
table, an entry for the flow label is created. If there is an 

35 entry for the flow label, an output port associated with the 
flow label is obtained. The intermediate network device then 
sends the packet to the output port. This continues at each 
intermediate network device until each packet of a given 
flow reaches the second network device. 

40 With these and other advantages and features of the 
invention that will become hereinafter apparent, the nature 
of the invention may be more clearly tmderstood by refer- 
ence to the following detailed description of the invention, 
the appended claims and to the several drawings attached 

45 herein. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a network suitable for 
practicing one embodiment of the invention. 

FIG. 2 is a block diagram of a router suitable for prac- 
ticing one embodiment of the invention. 

FIG. 3 is a block flow diagram of the steps performed by 
a first network device in accordance with one embodiment 
of the invention. 

FIG. 4 is a block flow diagram of the steps performed by 
an intermediate network device in accordance with one 
embodiment of the invention. 

DETAILED DESCRIPTION 

The embodiments of the invention are directed to enhanc- 
ing the consistency in service quality on the Internet. 
Whereas the soft-state and scalability have been key to the 
success of the Internet, its service quality is wanting. The 
65 embodiments borrow some of the concepts from connection- 
oriented networks, without compromising on the essential 
characteristics of IP. The embodiments are optimized for 
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carrying IP "flows/' and may be implemented as part of consistency) and in processing (admission control, and 

lower level (layer 2) protocols, including ATM or Multipro- negotiations). Thus, it is desirable that forwarding path of a 

tocol Label Switching (MPLS). One particular advantageous flow be stable. Stable forwarding paths also enable other 

embodiment of the invention creates the capability of run- optimizations. For example, though explicit routes can be 

ning the IP directly on top of the optical layer of a network. 5 specified in every IPv6 packet, significant bandwidth sav- 

TtieembodimentsoftheinventionprovideQoS capability can be accnied when stability of a flow path can be 

using the IP, particularly as set forth in Request For Com- "^Z'/'t^ ^1"!^ "^^^ T ^"T^ ' concept of route 

7 /xiT-r^\ TOGO J «T . *Ti„* 1 V/ rnru £\ piomng. loe same concept applies more generally to exten- 

ments (RFC) 1883 titled "Internet Protocol, Version 6 (IPv6) ^^^^ headers & / 

Specification/' dated December 1995 ("IPv6'0, which is ^^^^.^^ embodiments of the invention offer the 

mcorporated by reference herem By mtroducmg QoS char- 10 symmetric and reverse path routing. Symmet- 

acteristo usmg the IP, the need for carrymg IP datagrams m ^c routing is desirable because it simplifies algorithms and 

lower layer protocol umts A ATM ceUs) is avoided. This p^vides benefits to some services. Reverse path routing is 

reduces network complexity by removing a network layer, necessary for flow level control where it must be possible to 

which in turn simplifies network and service management. send control messages on the reverse forwarding path. Part 

Furthermore, using IP also avoids the complexity of map- 15 jj^^ complexity of RSVP is due to mechanisms to provide 

ping customer requests (at the network interface) onto a control path on the reverse data path. Explicit labeling of 

requests on the physical backbone infrastructure that may flows will enable routing symmetry, 

have conflicting service models. The embodiments of the invention provide the fundamen- 

The embodiments of the invention explicitly identify a tal strengths of connections while retaining the softness of 

group of IP packets as belonging to a "flow," and ensures ^ state and aggregationpropertiesof connectionless networks, 

that packets of a flow traverse the same sequence of routers. Th^ embodiments of the invention makes use of flows that 

This creates some unique advantages, such as load designated as "special" or "distinguished" through 

balancing, enhanced management and accountability, the .^^^ ^ ^^^^^ identifier or label for each flow. A 

ability to assign attributes to a flow, amortizing costly (in distmguished flow can then be pinned to a route siipport 

terms of bandwidth) operations over a sequence of packets, ^ '^i^ ""'^ associated (QoS) 

and providing symmetric and reverse path routing. ^ * , . , 

. ^ , J , . , J . , ■ , „ . A new now can be established by usmg a previously 

A first advantage is load balancing. In current IP, all the undeclared flow name. Assignment of a naine declares the 

packets from a source mtended for a given destination intent to use this flow for something special, and enables the 

traverse the same path. This technique does not optimally end-system to refer to the flow for later attribute assignment. 

utiUze the network resources. Furthermore, if a link on the An unknown flow name is interpreted as a request for a new 

path gets overloaded, a router may reroute either all or part flow. 

of the traffic to a given destination. This leads to either addition to the flow name, the first packet of a flow 

routing mstability or route fluttering, neither of which is (effectively the flow-rcquest) contains a datagram of a- 

desirable for a large class of appHcations. This can be network level protocol for which the network node can do 

overcome usmg "route pinning," Route pimiing involves routing (e.g., IPv6). Although subsequent packets may con- 

ensunng that packets for a particular fiow traverse the same tain arbitrary datagrams, softness of state is achieved when 

sequence of intermediate network devices (e.g., routers). the transferred packets are of such "known" protocol. In 

Route pmmng enables the network to better utilize its th^t case, if the state is lost, the next packet of the named 

resources and avoid the route-flappmg problem while ame- ^ fi^w is processed as ff it were the first packet of a new flow, 

horatmg routing mstability. optimize the transport network for IPv6, the network 

A second advantage is that the embodiments of the nodes simply support IPv6 routing. The flows defined herein 

invention enhance management and accountability within incur no call setup delay. Moreover, adopting (or assuming) 

the network. Recording of network usage, whether for a "use it or loose it" state invalidation policy, there is no need 

billing or off-line diagnostic analysis, is an important part of 45 for explicit tear-down. A flow may be uni- or bidirectional, 

providing network services. This is particularly important in There are four aspects of constructing a flow: (1) declar- 

a network that provides some assurance regarding quaUty. ing a name; (2) pimiing the route, (3) enabUng reverse path 

Labeled flows traversing a fixed path enable this function- routing, and (4) assigning attributes (such as QoS). 

^^^^y* ' Abstractly, current network nodes maintain two tables, a 

A third advantage is that certain attributes, such as QoS 50 routing table and a forwarding table. In the case of a 

attributes, can be assigned to designated flows. For a number traditional router the forwarding table corresponds to the 

of reasons, it may be desirable to assign attributes to routing cache. On an ATM switch or an MPLS Label Switch 

distinguished flows, for example to reserve resources along Router (LSR), the forwarding table is respectively the Vir- 

the path. This requires a mechanism to declare a path as tual Channel (VC) lookup table or the label lookup table. To 

"special" and then to describe (and possibly negotiate) the 55 support the flows used in the embodiments of the invention, 

path attributes. RSVP is designed for this purpose. The use a traditional router would be augmented with an additional 

of RSVP with the current IP (e.g., IPv4), however, is forwarding table for mapping flow names to flow state 

particularly complex without some mechanism to declare a (including the output port), as discussed in more detail with 

path as "special." This problem is overcome somewhat using reference to FIG. 2. 

the flow label already defined for IPv6. eo A flow request may be interpreted as an impUdt request 

A fourth advantage is that costly operations in terms of for route pinning. If not, route pinning may be requested 

bandwidth can be amortized over a sequence of packets. subsequently in a separate message. Without route pirmiog, 

Assignment of flow attributes (for example, reservations), the entry in the fiow cache simply points to the correspond- 

requires some mechanism to establish state and share that ing entry (in the regular cache) for the destination address, 

state across sequence of datagrams. State sharing fails if the 65 When route pinning is requested this entry is copied and thus 

forwarding path is not stable across multiple packets. Install- becomes independent of changes in the default destination 

ing state is expensive, both in latency (end-to-end based route. 
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Reverse path routing on a flow requires the node to record 
the incoming port as a part of the flow forwarding state. In 
addition this information must be conveyed to the output 
port. This given, however, reverse path routing may be 
achieved either by constructing a new path in the reverse 
direction (ie., a path association) or by a naming convention 
allowing a name to be resolved in the reverse path name 
space. 

Other flow attributes are signaled separately, and can be 
processed with specialized software using a conventional 
processor, or by a dedicated attribute control processor. 
Flows maintain a control mapping separate from the for- 
warding map, thus supporting control paradigms allowing 
service specific controllers, potentially installed on demand, 
to process the attribute messages. 

To aggregate state, a node (e.g., a backbone border node) 
may aggregate smaller flows and tunnel the aggregate flow 
to a particular node in the network (an egress router for 
example). After exiting the tunnel, the data packets would 
then be routed to their respective destinations as if they had 
originated at the tunnel end. To construct the tunnel, the 
router precedes the user packets with a "tunnel request" 
packet, a datagram of the "known" protocol, distinguished 
as a tunnel request. In the case of the IP as the known 
protocol, explicit routing could be specified using the source 
routing option (extension header for IPv6). Of course this is 
complementary to the use of normal tunneling, which them- 
selves might exploit flows as well. 

The IPv6 flow label can be used to implement a flow in 
an IPv6 network. To define a name an end-system sets a 
locally unique flow label on a packet intended for that flow. 
Hop-by-hop extension headers are used to assign attributes 
to the route. For example, the hop-by-hop router alert 
extension can be used, although additional extension head- 
ers are possible as weU. 

For network centric flows in IPv6, a router may identify 
a sequence of packets whose flow label is not set and 
aggregate them into a tunneled flow. The tunnel request 
packet is an IPv6 datagram carrying the assigned flow label. 
The router then sets the flow label of subsequent packets, 
which are then nullified on exit from the tuimel. To aggre- 
gate labeled flows we use traditional IP tunneling with flows. 

The embodiments of the invention modify the current 
definition of the IPv6 flow label for reverse path forwarding. 
The first bit of the flow label specifies whether it is source 
or destination unique, with a zero (0) implying a destination 
unique flow label, and a one (1) declaring a source unique 
flow label. To send on the reverse path, the receiver flips the 
first bit of the flow label. Since the source and destination are 
also swapped (as compared to the received packet), the same 
address is used with the flow label to uniquely identify the 
flow. 

Implementing the flow concept in MPLS is similar to that 
for IPv6. A sender uses a label to define a flow name. As 
labels are "link local" this amounts to upstream label allo- 
cation. Flow pinning is implemented as with IPv6. Attributes 
are signaled using hop-by-hop router alerts. Reverse path is 
accomplished in a manner similar to IPv6, that is, dividing 
the namespace on each liiik into two, with a direction 
distinguished by the leading bit. It is also possible, however, 
to use a separate flow name to associate the reverse path to 
the corresponding label path. Furthermore, this flow asso- 
ciation might be maintained only at the higher level, and not 
be explicit at the label path level. This could for example be 
the case if running IPv6 over MPLS. 

The embodiments of the invention aggregate separate 
flows to a tunnel in MPLS by using a label stack. A label is 
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pushed on the stack at the entry of the tunnel, and popped off 
on exit. As the tunnel end may in effect be a multiplexing 
point (i.e., a virtual termination of many tunnels) the tunnel 
label must be assigned by the tunnel termination node 
5 (downstream allocation), to ensure that the enclosed labels 
further down on the stack are uniquely resolved. To avoid 
subsequent round trip delays, however, the first request may 
yield two labels, allowing the entry point node to maintain 
a cache one label for subsequent tunnel construction. 
10 Referring now in detail to the drawings wherein like parts 
are designated by like reference numerals throughout, there 
is illustrated in FIG. 1 a network suitable for practicing one 
embodiment of the invention. FIG. 1 illustrates an exem- 
plary network 100 having multiple hosts and multiple inter- 
im mediate network devices connected as shown. Network 100 
shown in FIG. 1 represents one possible network configu- 
ration and wiU be used to describe the operation of the 
invention. Specifically, three intermediate network devices 
108, 110 and 112 are coupled to one another as shown. FIG. 
20 1 also illustrates three host devices 102, 104 and 106. Each 
host is coupled to a particular intermediate network device 
using an interface 120 (not shown). Interface 120 may be 
any type of interface circuit, including a network, capable of 
coupling one or more hosts to an intermediate network 
25 device. Alternatively, interface 120 may be omitted, and the 
host (or hosts) coupled directly to the intermediate network 
device. To simplify the illustration, only one host device is 
shown coupled to each intermediate network device. Those 
skilled in the art wfll appreciate that multiple hosts may be 
30 coupled to a single intermediate network device and a single 
host may be coupled to multiple intermediate network 
devices. 

It can be appreciated that the particular configuration 
shown in FIG. 1 is chosen as an example only and is not 
limitive of the type of network on which the present inven- 
tion can work. The number of configurations that networks 
can take are virtually limitless and techniques for setting up 
these configurations are well known to those skflled in the 
art. The embodiments of the present invention can operate 
^ on any of these possible configurations. 

Furthermore, both the host device and interaiediate net- 
work device can represent several types of devices. An 
example of a host device would be an end system (ES). An 
ES is a device attached to a network or subnetwork that is 
used to support end-tiser applications or services (e.g., a 
personal computer). An example of an intermediate network 
device would be a router, AIM switch or LSR. In this 
embodiment of the invention, a router is used as an example 
to demonstrate the principles described herein. Furthermore, 
the router utilizes IPv6 to route individual packets between 
hosts or end systems. 

FIG. 2 is a block schematic diagram of a router suitable 
for practicing one embodiment of the invention. Arouter 200 

55 is capable of incorporating the teachings of the present 
invention and includes a routing engine 202 having a 
processor 204 and a storage device 206. Storage device 206 
may be any suitable computer readable memory device, 
such as one or more dynamic random access memory 

60 (DRAM) devices, disk drives, or other mechanism for 
storing data. 

Routing engine 202 includes in storage device 206 vari- 
ous computer program segments that when executed by a 
processor (e.g., processor 204) performs the functionality 
65 for the various embodiments of the invention. In one 
embodiment of the invention, the computer program seg- 
ments are combined into a single flow management module 
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(FMM) 218. It can be appreciated, however, that the func- 
tions performed by this module can be separated into more 
modules, or be distributed throughout the system, and still 
fall within the scope of the invention. Furthermore, although 
this embodiment of the invention implements the function- 5 
ality of this module in software, it can be appreciated that the 
functionality of this module may be implemented in 
hardware, software, or a combination of hardware and 
software, using well-knowD signal processing techniques. 

Routing engine 202 includes also includes various tables lO 
208 and databases 210 contained within storage device 206. 
Tables 208 and databases 210 maintain information neces- 
sary for router 200 to properly forward data. Tables 208 may 
include a Routing Table and a Flow Table. Databases 210 
may include a Link State Database and a Forwarding 15 
Database. Routing engine 202 is capable of calculating paths 
through a network based on information contained in tables 
208 and databases 210, as well as the functionality provided 
by FMM 218. 

Input/Output (I/O) interfaces 212 are coupled to routing '^^ 
engine 202 and provide a physical connection to one or more 
network links 216. I/O interfaces 212 may be any suitable 
means for controlling communication signals between 
objects using a desired set of protocols, services and oper- 
ating procedures, such as IPv6. In this embodiment of the ^ 
invention, I/O interfaces 212 are bidirectional, that is, sig- 
nals can be sent and received using any particular I/O 
interface. Those skilled in the art, however, will recognize 
that uni-direction interfaces can also be used and fall within 
the scope of the invention. Furthermore, those skilled in the 
art will understand that the communication signals may be 
received over any suitable medium such as twisted-pair 
wire, co-axial cable, fiber optics, radio -frequencies, and so 
forth. 

Processor 204 may be any general purpose microproces- "'^ 
sor having sufBcient speed to implement the functionality 
described herein, such as the Pentium®, Pentium Pro, or 
Pentium II processors made by Intel Corporation. 

It can be appreciated that although router 200 is used as ^ 
an example to describe this embodiment of the invention, 
those skilled in the art will appreciate that various types of 
routers and other intermediary network devices may be used 
with the invention described herein. 

FIG. 3 is a block flow diagram of the steps performed by 45 
a first network device in accordance with one embodiment 
of the invention. As shown in FIG. 3, a packet for the 
information is generated at a first network device at step 302. 
Aflow label is assigned to the packet at step 304. A direction 
is assigned to the packet using the flow label at step 306, The 5Q 
packet is sent to a second network device through the 
network at step 308. 

With respect to step 306, an example of assigning a 
direction includes a flow label having a plurality of bits. The 
first network device would indicate the direction for the 55 
packet by modifying the flow label. The first bit of the flow 
label specifies whether it is source or destination unique, 
with a zero (0) implying a destination unique flow label, and 
a one (1) declaring a source unique flow label. To send on 
the reverse path, the receiving network device flips the first eo 
bit of the flow label. Since the source and destination are also 
swapped (as compared to the received packet), the same 
address is used v^th the flow label to uniquely identify the 
flow. 

FIG. 4 is a block flow diagram of the steps performed by 65 
an intermediate network device in accordance with one 
embodiment of the invention. As shown in FIG. 4, a first 
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packet is received at a first input port of an intermediate 
network device at step 402. A flow label for the first packet 
is identified at step 404. Whether a flow table has an entry 
for the flow label is determined at step 406. An entry for the 
flow label is created if no present entry at step 408. An 
output port associated with the flow label is obtained from 
the table at step 410. The first packet is sent to the output port 
at step 412. 

An entry for the flow label is added to the flow table if 
there is no present entry at step 408. Whether the flow label 
has an associated routing attribute is determined. A pointer 
associated with the flow label is stored in the flow table, the 
pointer pointing to an output port in a routing table for the 
intermediate network device, if the flow label does not have 
a routing attribute associated with it. An output port from the 
routing table is stored in the flow table associated with the 
flow label, if the flow label does have a routing attribute 
associated with it. 

The steps described with reference to FIG. 4 may be better 
understood using the foUowing example. In this embodi- 
ment of the invention, the intermediate network device is 
router 200. Router 200 receives a first packet at a first input 
port. FMM 218 of router 200 identifies a flow label for the 
first packet, FMM 218 searches a flow table stored with 
tables 208 to determine whether the flow table has an entry 
for die flow label. FMM 218 creates an entry for the flow 
label if no entry is currently in the flow table for the flow 
label. FMM 218 then obtains an output port associated with 
the flow label from the table. Routing engine 202 sends the. 
first packet to the output port. 

Router 200 performs the above steps with each packet that 
it receives. Packets received subsequent to the first packet 
and having the same flow label would be processed more 
e£Sciently since an entry for the flow label will already be 
present. If a packet does not have a flow label, FMM 218 can 
be programmed to ignore the packet in terms of processing 
it for flow control, or to assign a flow label if needed for 
more efficient routing (e.g., such as for tunneling). 

If there is no entry for the flow label present in the flow 
table, FMM 218 creates an entry. FMM 218 determines 
whether any attributes have been assigned to the flow label, 
such as route pinning. If a route pinning attribute has not 
been assigned to the flow label, a pointer associated with the 
flow label is stored in the flow table. The pointer points to 
an output port in the routing table for router 200. If a route 
pinning attribute has been assigned to the flow label, by 
FMM 218 or the first network device, FMM 218 copies the 
output port stored in the routing table having the same 
destination as the first packet in the flow table. The output 
port is associated with the flow label. In this manner, 
subsequent packets sharing the same flow label wiU traverse 
the path designated by the routing table if route pinning is 
not set, or wiU traverse the same path as the first packet if 
route pinning is set. In the latter case, a stable forwarding 
path is maintained since each packet within a flow will 
traverse the same sequence of routers. 

To enable reverse path forwarding, the first inpnt port 
where the first packet was received must also be stored in the 
flow table in association with the flow label for the first 
packet. If router 200 receives a second packet, FMM 218 
identifies a flow label for the second packet. Further, FMM 
218 also determines a direction for the second packet by 
examining the first bit of the flow label. If the first bit of the 
flow label indicates that the second packet is fi-om the second 
network device back to the first network device, then routing 
engine 202 sends the packet to the I/O interface port where 
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the first packet was initially received. This can be accom- As part of neighborhood discovery each node informs its 

plished since I/O interfaces 212 are bidirectional. Those upstream neighbor of the label space it is willing to accept, 

skilled in the art could appreciate that reverse path routing It is assumed that by standardization some small number of 

could also be implemented using unidirectional ports if labels may be taken for granted (say 1 or 10). The upstream 

necessary. 5 node may subsequently ask for the namespace to be 

The embodiments of the invention can also apply the expanded (this is done on signaling connections from 

aggregation properties already cxirrently tised in IPv4 and controller, to controller). 

IPv6 for individual packets to the concept of flows. For Labels are assigned for one-way connections by default, 

example, if router 200 receives a second packet at an input New extensions allow for bidirectional label assignment, 

port, it first identifies its flow label. Then it determines lO This can be done either by mechanisms to make a label valid 

whether the first packet and the second packet shoxdd be both ways, or alternatively by associating a new label path 

aggregated based on their flow labels and either associated going on a reverse route with a particular existing forward 

attributes or intelligence of routing engine 202. If the first path. 

and second packets should be aggregated, they are aggre- To create a new label path from a source (S) to a 

gated using conventional tunneling techniques. 15 destination (D), S creates a datagram (or otherwise the 

The use of flows in MPLS is similar to IPv6, with specific protocol transfer unit of the protocol used on the default 

modifications to take advantage of the underlying mecha- (signaling) path), aUocates a new label and issues a path- 

oisms for MPLS. This embodiment of the invention assumes request communicating a path request, the new label and the 

a "neighbor discover protocol" that can (be augmented to) datagram. 

carry capability information such as label space. It also '^^ The path request could be a new router alert option (an 

assumes that the network can do routing on demand inside extension header in IPv6), or could be communicated 

the label network (this is virtually implied by on demand implicifly either on a signaling channel, or by interpreting a 

routes). new (unknown) label as a path request. The last one offers 

There are two types of allocations in this embodiment of ^ particular advantages and is described in more detail below, 

the invention. The first is upstream allocated on-demand With respect to the new label, potentially more than a 

label paths of three sub-types: (1) without any acknowledg- single label will be pushed on the stack, for example when 

ments (acks); (2) with hop-by-hop ack*s; and for tunnels. constructing a tunnel. The new label(s) may be pushed on 

The second general allocation type is downstream allocated top of already existing labels. 

on-demand label paths. These are allocated using, for The datagram may encode information about routes (e.g., 
example, the methods and apparatus set forth in U.S. patent explicit routes), type of service, or desired service quality as 
application Ser. No. 09/015,496, filed on Jan. 29, 1998, the richness of the protocol on the default path allows, 
entitled "An Architecture For Ughtweight Signaling In ATM An intermediate node receiving a path-request processes 
Networks**, the entire disclosure of which is hereby incor- the message as follows. First, a new label entry is created in 
porated by reference. One of the down sides of downstream its forwarding table(s) unique on the pair (input port, label), 
allocation as compared to upstream albcation is that with This could be achieved by having a separate forwarding 
tunnels the hop-by-hop latency (over the tunnel) may be O table per input port. Note that input port here may be an 
(end-to-end) latency. One solution could be to place an IPv4 abstract input port, e., an end of a tunnel. The forwarding 
header into a single ATM cell. function on the enclosed datagram is then performed, yield- 
In both the upstream allocation and downstream 4q ing an output port determination. A new label is allocated on 
allocation, the label path setup can be accomplished in the the outgoing port. The label forwarding table is then 
following steps: (1) the end system initiates path setup; (2) updated, recording flie outgoing port and outgoing label. The 
the path (channel, featherweight flow); (3) can subsequently request is then forwarded to the appropriate output port. If 
assign additional parameters to the flow. One solution is to an acknowledgment is requested, then the intermediate node 
use a higher level protocol for routing and forwarding of 45 replies with an ack. If ack's are requested, the information 
Don-switched packets could be a semi-static and negotiated as part of neighbor 
As part of boot strapping nodes exchange capabilities and discovery. This could also, however, be done on demand by 
setup a default path that is subsequently used for out of band having the datagrams carry path construction attribute 
signaling. If a node imderstands multiple higher level pro- objects, 

tocols (i.e., can route using the rules of many protocols) e.g., 50 If a bidirectional path is requested then instead of replying 

IEV4, IPv6, ATM UNI, a different signaling path can be with an just an ack, a label is allocated a label on the reverse 

established for each of these protocols. path, the forwarding table is updated, the reverse label is 

The label distribution protocol uses a short label, which is recorded, and an ack is sent using the new upstream label, 

valid for a limited time and created on demand. This Some of the possible errors include "Reject — sent on a 

provides manageability, as the forwarding map can be 55 wefl-known (signaling) channel," This would also include a 

customized based on network conditions, the need of the reason code. 

flow, and so forth. Softness of slate is maintained for those To implement flows with a soft state, label paths can be 

protocol types for which a default path (and/or a signahng created for arbitrary data streams. For example, an access 

channel) is defined, and whose protocol data units carry node from a frame -relay network to a LSR network capable 

enough information to establish (recover) the forwarding 60 of routing IP datagrams only, would simply create a label 

state. This follows as the default paths per protocol assume path by creating and sending an IP datagrarn as a path 

that the LSR per protocol processing is suflScient to reach request, and encapsulating the frame -relay packets. When 

inside the MPLS encapsulation and parse the (header) new (unknown) labels are interpreted as label requests, 

information needed to (re)create the state. As a consequence, however, the state constituting labels paths used by well 

no explicit signaling is needed. The soft state is relatively 65 know protocols (i.e., one for which the LSR cloud can do 

efficient, as it is "use it or loose it," thus averting the need routing) becomes soft. This happens because the state may 

for keep-alive messages to maintain the state. be locally managed, and in particular lost, as the next 
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datagram arriving with a label that is no longer recognized 
will serve as a path request and effectively reconstruct the 
state. In this case there is no call setup delay. Moreover, by 
adopting (or assuming) a "use it or loose it" state invalida- 
tion policy, there is no need for explicit tear-down. 

RSVP is designed to support reservations for individual 
end-to-end flows on the Internet, in particular IPv4. An 
RSVP session is identified by a destination address and 
transport level protocol, and optionally the destination 
("generalized") port. A session is further classified into flows 
by receiver specified fillers. Whereas in principle these 
filters may be applied to any fields in the IP- or protocol level 
header (even potentially application level headers), current 
specifications and implementations of RSVP limit filters to 
a source address, and optionally the source port. Reserva- 
tions are on simplex streams and are exclusively receiver 
initiated. 

A destination address may be a multicast address, with the 
multicast session having multiple senders and receivers 
(multipoint-to-multipoint). Receiver initiated reservations 
may result in different reservations in different segments of 
the distribution of the multicast (variegated multicast trees). 
Moreover, in multicast sessions with multiple senders, 
receivers may msc the three different reservation "styles" 
(one of wild-card, fixed, or shared exclusive) to make 
reservations at even a finer level of the flow than specified 
by the filters. 

The two principal messages of QoS management in RSVP 
are the path message, sent from senders towards receivers, 
and the reservation messages, sent from receivers towards 
senders. Path messages establish flow identification state 
along the downstream path. This state includes filters and the 
trafiSc description (T-spec). Messages are processes at each 
hop before fonvarding. Reservations messages cany reser- 
vation requests (R-specs), and styles. Significant complexity 
is incorporated into RSVP to ensure that the RSVP signaling 
messages are forwarded to the same path (forward and 
reverse) as data is being forwarded. To make the RSVP state 
"soft" and to cope with route changes and changes in the 
topology of multicase distribution trees, path state and 
reservation state must be nefireshed periodically. 

Implementing RSVP using this embodiment of the inven- 
tion simplifies RSVP in several ways. First, this embodiment 
of the invention already does a flow classification, thus 
subsuming most of the filtering mechanisms of RSVP. In 
addition to the benefit of separation of mechanisms, using 
this embodiment of the invention allows for rich filtering at 
the edge of the network (to classify the incoming data- 
stream into flows), but very simple flow identification 
(explicit, or very trivial) inside the network. Filtering of finer 
grained subflows and RSVP reservation styles that apply on 
subflows (fixed, or shared exclusive) could be implemented 
by performing a nodal classification, or by defining a new 
flow for each of the subflows. This embodiment of the 
invention supports variegated trees similar to that of RSVP. 

RSVP messages are effectively signaled in -band on the 
established flow, distinguished with router alert option (hop- 
by-hbp extension header). Whereas this could be the stan- 
dard router alert options, the new "CC" extension header 
could also be used, further improving efficiency by allowing 
the RSVP messages to be forwarded on the output ports 
before nodal processing takes place. The latter is a departure 
from current RSVP semantics. Using this embodiment of the 
invention, the RSVP path message serves to advertise the 
T-spec, and possibly a filter for subflow classification. The 
previous hop information is not needed. As the message is 
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forwarded without processing the "Adspec" does not serve 
a ijseful purpose in this case. The reserve messages, 
however, must be processed at every branch point of a 
multicast flow. This is achieved using bidirectional flows 
5 and a blocking router alert option. Point-to-point 
connections, however, can exploit the in-band signaling for 
reservation messages, thus allowing the reservations to be 
processed in parallel. Other RSVP messages, for errors, 
tear-down, and confirmation are processed by the RSVP 
processor in a standard manner, but are transmitted in-band 
on the established flow. The RSVP processor must in addi- 
tion process a tear-down commands firom the forwarding 
engine, to invalidate reservation state for flows that have 
become invalid (at the forwarding level). 

Using the support for bidirectional flows, forwarding on 
the reverse path may be moved out of the RSVP control 
process, and handled at forwarding level. To aQow for 
softness of the reservation state and to allow for adju^ments 
in reservations in multicast flows as membership (and thus 
topology) changes, state refresh may stiff be needed. The 
stabiUty of the connections, however, can be exploited and 
flierefore the need for frequent state refresh may be reduced. 
In particular, it is feasible to have the data traffic refresh the 
state, limiting refresh only to "keep-alives" during extended 
inactivity. This is particularly true for point-to-point flows, 
^ The need for state refresh is further reduced if the route of 
the flow is pinned. Therefore, removing the connectivity 
issues from the RSVP processing, and benefiting from the 
StabiUty caused by connections, RSVP is simplified and may 
be more optimized for common cases (e.g., point-to-point 
flows) whfle retaining the essential qualities of RSVP. 

It is worthy to note that any reference in the specification 
to "one embodiment" or "an embodiment" means that a 
particidar feature, structure, or characteristic described in 
connection with the embodiment is included in at least one 
embodiment of the invention. The appearances of the phrase 
"in one embodiment" in various places in the specification 
are not necessarfly aU referring to the same embodiment. 

Although various embodiments are specifically iUustrated 
^ and described herein, it will be appreciated that modifica- 
tions and variations of the present invention are covered by 
the above teachings and within the purview of the appended 
claims without departing from the spirit and intended scope 
of the invention. For example, although a router was used in 
certain embodiments of the invention, those skiUed in the art 
will appreciate that the principles described herein can also 
be applied to other network devices such as ATM switches 
or LSRs. 

What is claimed is: 

1 . Amelhod for communicating information in a network, 
comprising: 

generating a packet for the information at a first network 
device; 

assigning a flow label to said packet; 
55 assigning a direction to said packet using said flow label, 
and wherein said flow label is reverse path forwarding 
enabled; and 

sending said packet to a second network device through 
the network. 

60 2. The method of claim 1, wherein said flow label is 
comprised of a pluraUty of bits, and wherein said step of 
assigning said direction comprises the step of setting one of 
said bits in said flow label. 

3. A method for communicating information in a network, 
65 comprising: 

recei%dng a first packet at a first input port of an interme- 
diate network device; 
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identify ing a flow label for said first packet; 
determining whether a flow table has an entry for said 
flow label; 

creating an entry for said flow label if no entry is present; 
obtaining an output port associated with said flow label 

from said table; and 
sending said first packet to said output port. 

4. The method of claim 3, wherein said step of creating an 
entry comprises the steps of: 

adding an entry for said flow label to said flow table; 

determining whether said flow label has an associated 
routing attribute; 

storing a pointer associated with said flow label in said 
flow table, said pointer pointing to an output port in a 
routing table for said intermediate network device, if 
said flow label does not have a routing attribute asso- 
ciated with it; and 

storing said output port from said routing table in said 
flow table associated with said flow label, if said flow 
label does have a routing attribute associated with it. 

5. The method of claim 3, further comprising the step of 
storing said first input port in said flow table associated with 
said flow label. 

25 

6. The method of claim 5, wherein said input and output 
ports for said intermediate network device are bidirectional, 
further comprising the steps of 

receiving a second packet at a second input port of said 
intermediate network device; 30 

identifying a flow label for said second packet; 

determining a direction for said second packet using said 
flow label; and 

obtaining said first input port associated with said flow 
label from said flow table; and 

sending said first packet to said first input port. 

7. The method of claim 3, further comprising the steps of 
receiving a second packet at an input port of said inter- 
mediate network device; 40 

identifying a flow label for said first packet; 
determining whether said first and second packets should 

be aggregated; and 
aggregating said first and second packets in accordance 

with said determination, 

8. A method for requesting a connection-oriented packet 
flow between a first network device and a second network 
device in a packet network, comprising 

allocating a flow label to the packet flow that has not been 
previously declared; 

assigning the flow label to a header in a first packet of tbe 
packet flow; 



adding any desired flow attributes to the header, and 
sending the packet to the second network device through 
the network, where in the flow label that has not been 
previously declared is interpreted by the second net- 
work device as a request for a new packet flow and the 
second network device can utUizes the flow attributes 
in establishing state for the packet flow, and wherein 
the flow label includes an indication of direction for the 
flow. 

. 9. The method of claim 8 wherein the second networic 
device treats the flow label as a request to pin the route 
between the first network device and the second network 
device. 

10. The method of claim 8 wherein the second network 
device may send another packet in a reverse path by using 
a second flow label including a different indication of 
direction for the flow. 

11. The method of claim 10 wherein the flow attributes 
reflect quality of service attributes for the packet flow. 

12. A network router comprising: 
a processor; 

a plurality of input/output (I/O) interfaces connected to 
the processor; and 

one or more storage devices, connected to the processor, 
further comprising a routing table, a flow table, and a 
computer program which when executed by the pro- 
cessor performs a method of establishing a packet flow 
between the network router and a second network 
router comprising the steps of: 
receiving a packet from the second network router with 

a flow label that has not been previously declare; 
interpreting the flow label as a request for a new packet 
flow between the network router and the second 
network router; and 

creating an entry for the flow label in the flow table, and 
wherein the flow label includes an indication of direc- 
tion for the flow. 

13. The network router of claim 12 wherein the request for 
a new packet flow is treated by the network router as an 
implicit request for route pinning between the network 
router and the second network router. 

14. The network router of claim 12 wherein the network 
router may send another packet in a reverse path to the 
second network router by using a second flow label includ- 
ing a different indication of direction for the flow. 

15. The network router of claim 14 wherein the packet 
includes quality of service attributes for the packet flow and 
wherein the network router can establish state based on the 
quality of service attributes. 
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